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INTEGRATED  DATA  AND  CONTROL  LEVEL  FAULT  TOLERANCE 
TECHNIQUES  FOR  SIGNAL  PROCESSING  COMPUTER  DESIGN 

G.  Robert  Redinbo 

I.  INTRODUCTION 

High-speed  signal  processing  is  an  important  application  of  special  purpose 
numerical  processor  designs.  These  digital  processing  systems  involve 
interconnecting  complex  integrated  circuits  or  a  large  self-contained  VLSI  design;  both 
implementations  contain  dense  electronic  structures  susceptible  to  internal  temporary 
and  permanent  failures.  Such  systems  are  difficult  to  protect  sufficiently  by  classical 
fault-tolerant  computer  design  methods  [1]  because  they  employ  irregular  electronic 
configurations  embedded  in  otherwise  extremely  regular  structures.  Nevertheless,  the 
principal  function  of  signal  processing  is  to  manipulate  numerical-based  data,  and  the 
only  errors  of  concern,  whether  affecting  control  or  data  operations,  are  those 
corrupting  the  output  numerical  data. 

The  work  examined  new  methods  for  introducing  efficient  fault-tolerance  design 
methods  in  linear  signal  processing  systems  without  degrading  high-speed 
performance.  The  main  goal  was  to  develop  new  practical  approaches  for  realistic 
high  speed  designs.  The  fundamental  concept  relies  on  a  Watchdog  Parity  Processor 
that  observes  both  input  and  output  numerical  samples  and  computes  a  few  parity 
values  efficiently  (see  Figure  1-1).  This  sparse  number  of  parity  values  is  enough  to 
judge  if  the  main  processor  is  functioning  properly  in  the  presence  of  internal 
temporary  or  permanent  errors,  even  in  the  watchdog  unit  itself.  Only  the  detection  of 
errors  is  sufficient  for  protection  since  error  correction  techniques  could  easily  require 
as  many  resources  as  the  original  processing  system.  It  is  more  efficient  to  recompute 
a  series  of  calculations  than  to  try  to  correct  them. 

This  approach  to  fault-tolerant  signal  processing  is  representative  of  a  general 
class  of  protection  techniques  called  algorithmic  fault-tolerance  [2-7].  The  class 
derives  its  name  from  the  fact  that  certain  redundant  properties  attached  to  a  numerical 
processing  algorithm  are  used  to  check  the  execution  and  output  of  the  algorithm.  The 
redundancy  may  be  introduced  in  the  basic  algorithm  using  numerical  parity  codes 
[2, 3,8, 9]  or  may  occur  naturally  in  certain  forms  of  an  algorithm,  e.g.,  matrix  equation 
solution  methods  [10]. 

Algorithmic  fault-tolerance  protects  a  potentially  very  wide  class  of  internal 
errors.  The  classical  stuck-at  type  of  faults  and  soft  errors  are  easily  covered  if  they 
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manifest  themselves  at  the  numerical  level.  Numerous  errors  in  a  whole  subassembly 
of  a  design  may  influence  only  a  few  numerical  samples  wher«  the  numerical-based 
redundancy  will  detect  them.  Furthermore,  algorithmic  fault-tolerance  is  robust  since  it 
is  less  sensitive  to  many  low-level  architectural  defects.  This  approach  obtains  its 
requirements  from  the  numerical  input-output  relationship  imposed  by  the  signal 
processing  function. 

New  protection  techniques  are  now  possible  due  to  the  relatively  recent  results 
concerning  real  parity  codes  [7,8,9,11,12].  Both  block  and  convolutional  codes  have 
been  examined.  The  block  codes  can  be  very  powerful  generalized  cyclic  codes 
which  have  the  best  minimum  distance  structure  achievable  by  a  linear  code  [11,12]. 
The  first  class  of  convolutional  codes  discussed  by  Marshall  [1 1]  were  based  on  binary 
codes  viewing  the  binary  elements  as  having  counterparts  in  the  real  field.  However, 
other  classes  are  possible.  It  is  easy  to  define  a  type  based  on  real  cyclic  codes  using 
an  old  construction  technique  of  Wyner  [13,  Section  13.3].  Mathys  [14]  has  another 
construction  for  rate  \  real  convolutional  code  based  on  dual  cyclic  codes  (see  Section 
11.3  in  [15]  for  theory).  Regardless  of  the  exact  real  code,  performance  and  speed 
constraints  in  practical  signal  processing  implementations  dictate  that  any  protection 
method  cannot  disrupt  or  alter  the  original  function  beina  protected.  This  implies  that 
systematic  encoding  forms  of  the  codes  are  required  [13,15].  This  in  turn  insures  that 
real  convolutional  codes  are  noncatastrophic  [1 5]. 

The  next  section  separates  the  type  of  signal  processing  into  two  classes 
depending  on  the  length  of  the  input  data  segment  processed.  The  linear  signal 
processing  is  related  to  a  finite  or  infinite  matrix  according  to  the  class.  The  generation 
of  real  parity  samples  is  discussed  and  the  necessary  notation  is  established.  The 
third  section  introduces  the  role  of  the  mean-square  error  (MSE)  between  comparable 
parity  values.  Internal  hardware  errors  and  roundoff  and  quantization  noise  are 
permitted  in  the  main  processor  hardware  as  well  as  in  the  subassemblies  that 
compute  and  compare  the  parity  samples.  The  optimum  parity  estimators  are 
developed  and  the  minimum  mean-square  error  expressions,  both  for  finite  and  infinite 
processing  lengths  are  given. 

The  following  section  presents  techniques  for  simplifying  the  computational 
demands  in  the  Watchdog  Parity  Processor.  The  efficiencies  are  based  on  the 
algebraic  structures  of  the  real  parity  codes  employed.  For  real  cyclic  codes,  the  parity 
samples  attached  to  finite  length  convolution  are  easily  calculated  from  the  input 
parities.  On  the  other  hand,  when  infinitely  long  input  samples  are  being  processed, 
the  real  convolutional  code’s  structure  may  be  modified,  without  degrading  the  error 
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protection  level,  to  reduce  the  parity  computation  rate.  In  this  way  the  input  data 
samples  are  used  less  frequently  in  generating  the  required  parity  samples. 
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II.  PARITY  PROTECTION 


The  linear  signal  processing  system  to  be  protected  may  be  described  by  a 
matrix  F  relating  the  input  data  in  a  vector  u  to  the  output  data  in  vector  y.  The  size  of 
the  matrix  and  respective  vectors  may  be  finite  or  infinite  depending  on  the  data  being 
processed. 

y  =  UF  (2-1) 

This  view  encompasses  several  situations  including  standard  digital  filtering  using 
either  finite  impulse  response  (FIR)  or  infinite  impulse  response  (MR)  weighting.  Such 
a  model  does  not  constrain  the  actual  implementation  details.  However,  the  exact 
impact  of  roundoff  noise  and  soft  errors  on  svstem  reliability  is  directly  related  to  this 
computational  configuration.  Infinite  arithmetic  precision  will  be  assumed,  but  the 
effects  of  roundoff  and  quantization  noise  will  be  included  through  a  probability  density 
function  describing  the  statistical  behavior  of  the  model. 

Error  protection  './ill  derive  from  employing  parity  samples  dictated  by  real 
number  codes.  The  existence  of  powerful  real  number  codes  (including  maximum 
distance  separable  ones)  is  one  reason  for  choosing  this  approach.  Furthermore,  the 
necessary  parity  values  may  be  formed  in  parallel  with  the  normal  data  processing 
operations  so  that  there  is  no  speed  degradation,  a  major  requirement  in  fault-tolerant 
signal  processing  systems. 

A  general  protection  scheme  is  shown  in  Figure  2-1.  As  every  group  of  k  input 
and  output  samples  is  processed  by  the  linear  system,  described  by  matrix  F,  two  sets 
of  ( n  -  k)  parity  values  are  produced  respectively  at  the  input  and  output. 
Correspondingly  related  parity  samples  are  checked  in  a  comparator  to  detect  any 
system  errors.  In  the  case  where  the  system  processes  blocks  of  data,  the  code  is  a 
real  number  block  (n,k)  code  [1 1,12].  On  the  other  hand,  if  the  input  is  processed  as  a 
continuous  infinite  sequence,  a  real  convolutional  code  [7,11]  is  employed.  Generally, 
in  either  case  the  number  of  parity  values  is  kept  small  to  make  the  protection 
overhead  manageable. 

This  error  detection  philosophy  derives  from  experience  with  finite  field  based 
error  detection  schemes.  The  parity  values  are  determined  by  the  real  code  applied  to 
the  output  samples  while  their  respective  estimators  are  formed  in  the  Parity 
Estimation  subsystem  shown  in  Figure  2-1  where  the  effects  of  F  and  the  parity 
equations  are  combined. 

The  distance  properties  of  the  real  code  are  used  to  detect  if  errors  within  the 
implementation  have  caused  any  significant  differences  between  the  desired  and 
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actual  computation.  It  is  assumed,  as  is  the  usual  case  in  fault-tolerant  computing 
system  design,  that  errors  occur  in  only  one  subassembly  of  Figure  2-1  at  a  time:  in 
either  the  parity  estimator,  main  processing  unit,  parity  constructor,  or  the  comparator. 
The  approach  discussed  here  permits  errors  in  any  of  there  parts,  but  the  heavy 
concentration  of  computational  effort  in  the  main  processing  section  suggests  a  higher 
likelihood  of  errors  there. 

The  parity  estimation  part  necessarily  appears  to  duplicate  the  effects  of  F. 
However,  later  results,  particularly  in  important  practical  cases,  will  demonstrate  how 
these  estimations  may  be  done  quite  efficiently  without  doubling  the  computational 
effort.  Nevertheless,  the  real  number  code  parities  are  checked  in  the  comparator. 
Since  roundoff  and  quantization  noise  may  be  present  even  in  the  error-free  situation, 
a  small  comparison  threshold  must  be  allowed  in  determining  if  two  differently 
calculated  parities  mismatch  due  to  internal  hardware  errors. 

An  accepted  method  of  analyzing  roundoff  and  quantization  errors  in  linear 
processing  systems  employs  first  and  second  order  statistical  moments.  This  analysis 
basically  considers  the  mean-square  average  of  the  difference  between  ideal  results 
and  corresponding  values  when  roundoff  and  quantization  effects  are  preser..  t :  5-20]. 
The  mean-square  error  is  prevalent  in  digital  filtering  and  communication  systems  [20- 
22].  The  original  motivation  for  error  detection  uses  the  minimum  probability  of  error 
criterion  [13,22],  but  usually  roundoff  and  quantization  errors  are  considered  second- 
order  effects  and  do  not  influence  the  design  of  the  error-detecting  subsystem.  It  is 
natural  to  view  the  detection  of  errors  by  comparing  parity  samples  as  a  probability  of 
error  motivation,  while  simultaneously  minimizing  the  mean-square  error  (MSE) 
between  comparable  samples.  This  dual  concept  will  be  explored  in  the  next  section 
where  real  error-detecting  codes  with  good  separation  properties  will  govern  the  parity 
construction  subassembly  while  the  parity  estimation  part  is  designed  to  minimize  the 
mean-square  error  between  respective  parity  samples. 

The  finiteness  of  the  size  of  the  linear  processing  matrix  determines  the  type  of 
parity  code  that  is  employed.  For  finite  length  inputs,  a  systematic  real  linear  block 
code  will  be  applied,  whereas  if  the  input  data  represent  a  continuous  stream,  a 
systematic  real  linear  convolutional  code  will  be  used.  The  finite  processing  case  with 
a  block  code  will  be  examined  first,  and  after  some  preliminary  notation  is  established, 
the  infinite  length  case  will  be  addressed.  This  notation  and  setting  will  be  the  basis 
for  the  MSE  estimation  results  developed  in  the  next  section. 

The  linear  signal  processing  operation  for  finite  length  data  inputs  will  be 
represented  by  the  (kxk)  matrix  F.  The  input  data  are  denoted  by  the  (1  xk)  vector  u. 
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though  its  tail  may  actually  contain  padding  zeros  because  the  input  is  really  shorter 
than  k.  The  output  data  in  vector  y  are  obtained  through  a  linear  equation.  All 
indexing  starts  with  0  to  be  consistent  with  a  concept  of  time  relating  to  index  position. 


(v0,  vv  ....  v^)  =  y  =  uF  :  U  =  {uQt  U) . uM) 

'  4,0  4,1  4,*-1 

fX0  fXl  —  ^I.A-1 

F  = 

•  •  •  • 

•  •  •  • 

•  •  •  • 

Jk-X0  4-1,1  '•*  4-Vr-l 


(2-2) 


This  more  general  case  includes  the  special  situation  of  convolving  the  input  data  in  u. 
length  t,  with  a  finite  impulse  response  represented  by  w,  length  s,  where  s  +  t£k. 

11  =  ( u0 ,  u, . uM,0, 0, ....  0) 

*  =  (4-  4-  *2 . 4-1 ) 

Vq  Wi  •••  W5_-|  0 

0  W0  —  Ws_2  Ws_-,  - 

•  •  •  •  •  • 

Jflf  ■  (Wa,  wj,  w-i,  °,  0, ....  0)  =>  F  =  :  ;  '  ‘ 

o  o  o  . 

•  «  •  *  *  • 

•  •  •  »  «  « 

•  ■  •  •  •  • 

0  0  •••  0  ••• 


M 

4  =  r  =  0,1 . fr-1  (2-3) 

i~0 

The  output  data  contained  in  y  will  have  parity  associated  with  it.  The  (n-k)  parity 
values  are  determined  by  a  block  code  whose  generator  matrix  takes  the  special 
systematic  form. 

G  *  (lk  |  O)  ;  lk  is  a  kxk  identity  matrix. 

The  parity-check  submatrix,  (2-4) 
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O  =  ((cty))  ;  A=0. 1. ....  Ac-1 

j=  0, 1 . r>-k-1 . 

The  output’s  parity  values  are  dictated  by  the  parity-check  part  of  this  matrix  according 
to  the  equation 


k- 1 

=  JsQijvi  •  y  =  O.t . n-k-t  (2-5) 

/= o 

By  design,  the  real  code  has  good  distance  separating  properties  [11],  and  so  any 

errors  appearing  in  the  collection  of  positions  of  y.  combined  with  a0(y),  a.,(y) . 

a^.^y)  can  be  detected.  However,  section  3  will  examine  how  the  parity  choices  at 
the  output  y  affect  the  parity  estimation  subassembly,  Figure  2-1 . 

On  the  other  hand,  when  the  signal  processing  involves  an  infinitely  long  input 
data  stream,  the  parity  samples  will  be  determined  by  a  real  systematic  convolutional 
code.  One  reasonable  constraint  must  be  imposed  to  avoid  an  infinite  delay  before 
any  output  sample  appears.  The  output  vr  in  output  stream  y  where  r=  0,  1,  ...  only 
depends  on  input  values  u0,  uv  ....  ur  in  the  input  stream  y.  In  other  words,  the  linear 
signal  procesjing  matrix  F  has  zeros  below  entry  r  in  column  r.  This  causality 
constraint  can  be  relaxed  somewhat  by  allowing  a  finite  number  of  nonzero  samples 
beyond  item  r,  but  this  introduces  additional  delay  that  is  difficult  to  implement  in 
practice. 

The  encoding  matrix  for  a  systematic  convolutional  code,  G,  has  a  block  type 
format  involving  m  fundamental  finite  sized  matrices  whose  dimensions  are  related  to 
the  rate  and  number  of  parity  check  positions  in  the  code.  The  parameter  m 
determines  the  constraint  length  of  the  code. 
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Gm- 1 


0 

Gm 
Gm- 1 


(2-6) 


Each  kxn  submatrix  G,  has  a  distinctive  form. 


Go  =  (/|  Qq) 

;  / 

kxk  Identity  Matrix 

(2-7a) 

^0 

kx(n-k)  Parity-Check  Matrix 

Gj  -  (0  |  Qj) 

;  o 

kxk Zero  Matrix 

(2-7b) 

pi 

kx(n-k)  Parity-Check  Matrix 

7=1. 

2, ....  m. 

The  entries  in  the  parity  check  submatrices  Q-,  may  be  either  0  or  1  even  for  the  real 
Marshal  code  case  [1 1],  or  in  the  more  general  case,  real  numbers  [7,14]. 

The  parity  positions  are  a  function  of  possibly  {m  +  1  )k  input  samples  through 
the  action  of  the  Qj  parts  of  each  Gj.  The  stack  of  these  parity  weighting  values  will  be 
denoted  by  an  {( m  + 1  )kx  (n  -  k)}  matrix  Q  with  respective  columns  {  qr }: 
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(  o  \ 

um 

Qm- 1 


Q  = 


=  (OfrOySl  •••••SodLri)- 


02 

Ol 

,  °o  , 


Qc_  =  ((Qc05))  ;  y-0.1.2 . [(m  +  1)/c-  1]. 

qc  (m  +  1  )/c  x  1  Column  Vector 
c  =  0, 1,2, ....  ( n-k - 1) . 


(2-8a) 


(2*8b) 


The  indexing  in  the  parity  columns  of  G  is  a  traditional  one  which  shows  parity  values 
as  the  output  of  an  (FIR)  filter.  The  parity  samples  are  obtained  as  ( n  -  k)  convolution 
operations  on  the  output  data.  Employing  the  notation  of  equation  (2-8),  the  (n  -  k) 
respective  parity  channels  determine  the  /*h  parity  sample  in  stream  c  by  the  formula: 


where 


M  =  (m  +  1)* 

yM  =  (vq,  v1t ....  v}  ;  output  data  stream  up  to  item  vr. 


(2-9) 
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III.  MINIMUM  MEAN-SQUARE  ERROR  PARITY  ESTIMATION 


The  parity  estimation  subassembly  associated  with  the  real  code  parities 
appearing  at  the  processing  output  will  be  determined  in  this  section.  The  case  of 
finite  dimensional  signal  processing  is  examined  first  because  of  its  slightly  simpler 
notation.  The  parity  values  a 0&),  a^y),  ....  (y)  are  dictated  by  the  real  sym¬ 

metric  block  code,  e.g.,  equation  (2-5).  The  inherent  distance  property  of  this  code 
guarantees  that  if  any  reasonably  significant  errors  occur  in  a  number  of  samples, 
checking  the  parity  values  within  error  threshold  limits  will  detect  them.  However,  such 
a  detection  approach  is  still  true  if  the  parity  estimation  subassembly  presents  values 
whose  calculations  are  directed  by  a  mean-square  error  criterion.  The  code  is 
designed  for  good  probability  of  detection  whereas  the  parity  estimation  is  selected  to 
minimize  the  mean-square  error  between  related  samples.  One  criterion  has  gross 
errors  in  mind  while  the  MSE  focuses  on  roundoff  and  quantization  effects. 
Nevertheless,  each  criterion  considers  the  effects  of  all  errors;  there  is  a  different 
emphasis  depending  on  the  type  of  error. 

The  general  model  in  Figure  2-1  permits  hardware  errors  as  well  as  roundoff 
and  quantization  errors  in  each  of  the  three  computational  units;  signal  processing, 
parity  construction  and  parity  estimation.  The  mean-square  error  criterion  is  applied  to 
the  sum  of  the  squared  differences  between  the  parity  values  and  their  estimated 
counterparts.  This  overall  criterion  is  denoted  as  e2. 

e2=  £  E{|a,(!£)-p,(yf}  (3.,) 

The  expectation  uses  the  distribution  governing  the  elements  in  the  input  vector  u  as 
well  as  the  stationary  occurrence  of  hardware  errors  and  roundoff  and  quantization 
noise  in  the  three  computational  subassemblies. 

The  role  of  the  totally  self-checking  comparator  is  to  detect  significant 
differences  exceeding  the  threshold  A  between  individual  components,  including 
failures  and  errors  within  its  parts.  This  is  a  generalization  of  the  totally  self-checking 
equality  checker  used  in  fault-tolerant  computing  designs  [1 ,23].  One  conceptual 
implementation  appears  in  Figure  3-1  a  which  displays  the  comparison  process  for  a 
generic  position.  The  output  represents  a  1  out  of  2  code  whenever  the  two  inputs  are 
within  a  difference  magnitude  of  A.  Note  that  this  slice  is  self-testing  in  that  both  types 
the  1  out  of  2  code  words  appear  during  normal  operation  [23].  The  1  out  of  2  code 
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1  OUT  OF  2  CODE  OUTPUTS 
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Pi(u) 


.  f  5R5R 
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0,+A 


dj  =  otj(i()  — Pj&j) 


JO  IF  ej<0 
[1  IF  ej>0 

OUTPUT  IN 
1  OUT  OF  2  CODE 

0  IF  dj<J-A 
1  IF  -  A  <  dj  £  0 
~  0  IF  0<dj^+A 
1  IF  +A<di 


I  «j(Y) 

FIGURE  3-1  a  POSITION  COMPARATOR 


COMPARATOR  0 
OUTPUT 
COMPARATOR  1. 
OUTPUT 


1  OUT  OF  2  CODE 

CONCENTRATOR 


COMPARATOR  n-k- 
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FIGURE  3-1  b  COMBINING  POSITION 
COMPARATORS 

TOTALLY  SELF-CHECKING  COMPARATOR 


FIGURE  3-1 
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OUT  OF  2  CODE 

ONCENTRATOR 

OUTPUT 


outputs  from  each  of  the  ( n  -  k)  comparator  slices  are  concentrated  to  a  final  1  out  of  2 
output  as  shown  in  Figure  3-2b.  This  concentrator  is  a  binary  logic  device  which  is 
code  disjoint  [1 ,23]  and  totally  self-checking  in  the  usual  sense. 

In  order  to  understand  the  role  of  the  mean-square  error  criterion  on  the  exact 
structure  of  the  parity  estimator  subassembly,  a  simpler  situation  will  be  examined  first. 
All  errors,  whether  hardware  or  roundoff  and  quantization  errors,  will  be  confined  to 
the  signal  processing  part  only.  Most  of  the  techniques  and  mathematical  approaches 
will  be  expanded  later  to  the  situation  where  hardware  errors  and  roundoff  and 
quantization  noise  affect  all  computational  parts  simultaneously.  Furthermore,  in  this 
beginning  step  the  linear  signal  processing  subsystem  will  be  taken  as  finite  weighting 
processing  without  loss  of  generality. 

Finite  Length  Processing 

The  errors  in  the  signal  processing  part  will  be  described  using  a  conditional 
density  function.  (Degenerate  forms  of  errors  are  incorporated  in  this  model  by 
permitting  generalized  functions,  e.g.,  Dirac  delta  functions,  in  the  density  function.) 
This  multidimensional  conditional  density  function  is  denoted  by  ^U(y/u).  Since  it  is 

ERROR  AND  NOISE  EFFECTS 
DESCRIBED  BY  CONDITIONAL  DENSITY  FUNCTION 


common  practice  to  assume  noise  in  digital  signal  processing  systems  can  be 
modeled  realistically  by  additive  disturbances  [16-19],  this  conditional  density  function 
will  be  imbued  with  a  generalized  additive  property.  For  every  &  in  the  input  space, 
there  is  another  1  x  k  vector  £(&)  such  that  the  conditional  density  obeys  the  following 
law  for  all  input  vectors  u  and  output  vectors  y. 

+ £(x)/u  +  x)  (3-2) 
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The  implications  of  this  generalized  additivity  model  can  be  developed.  One 
candidate  for  x  is  -u,  and  so  the  effects  of  errors  are  typified  by  passing  the  all  zero 
vector,  denoted  by  Q,  through  the  signal  processing  subsystem. 

^/U(y/u)  =  +  5(-u)/Q)  (3-3) 

The  individual  vectors  £(x)  as  *  is  varied  define  a  transformation  from  the  input  space 
to  the  output  space.  It  is  natural  to  explore  aggregate  effects  of  this  transformation. 
Because  property  (3-2)  holds  for  all  inputs  u.  and  outputs  &  it  is  easy  to  show  the 
following  additive  properties  of  the  transformation  £(*). 

5Hl)  =  -£(il)  (3-4) 

5(a  +  b)  =  5(2)  +  5(b)  2,  b  input  vectors 

Thus  the  generalized  additive  assumption  leads  to  a  vector  transformation  £  that  has 
homomorphic  properties  [24].  Consequently,  it  is  reasonable  to  infer  that  the 
transformation  is  the  signal  processing  operation. 

£(U)  =  iiF  (3-5) 

The  code  dictates  the  parity  construction  rules  {a,<y)},  equation  (2-5),  while 
minimizing  the  mean-square  error  expression  (3-1)  will  govern  the  choice  of  the 
corresponding  parity  estimation.  It  will  come  as  no  surprise  that  the  optimum  choices 
for  the  parity  estimator  functions  are  the  respective  conditional  means  [20,25]. 

Py{y)  =  Ev/y{<*/(v)/u}  /  =  0,\...,(n-fr-1)  (3.6) 

The  conditional  expectations  employ  the  conditional  density  function  ^(y/u) 
described  above.  The  minimum  mean-square  error  corresponding  to  conditional 
mean  estimators  is  given  by  e2. 


x2 


=  "l'1{Ey[Ev/4l“/<^)|2/«}]  -  Ey[|P/y)f]} 


(3-7) 
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A  simple  proof  of  these  two  results  are  included  in  a  footnote  for  the  sake  of 
completeness.1 

A  vector  of  optimum  parity  estimators  with  respective  components  given  by 
equation  (3-6)  may  be  written  employing  matrix  and  vector  notation,  particularly  the 
parity-check  part  Q  of  equation  (2-4). 

P(y)  =  Ev/u{vO/u}  ;  Q  is  the  k  x  (n  -  k)  parity -check  part.  (3-8) 

The  conditional  expectation  may  be  simplified  by  considering  the  generalized 
additivity  property  of  the  underlying  conditional  density  function,  equations  (3-4)  and 
(3-5).  A  vector  integral  notation  will  be  used  to  express  the  conditional  expectation. 

P(y)  =  [Jyfv/y(v/y)dy]o. 


A  straightforward  change  of  variables,  z  =  y-U^.  permits  this  to  be  rewritten. 


1  A  calculus  of  variations  argument  to  the  proof  of  the  optimality  of  equations  (3-6)  and 
(3-7)  above  avoids  any  delicacies  that  might  arise  employing  a  differential  approach 
to  minimization.  Suppose  that  there  are  ( n  -  k)  other  functions  y/li)  with  finite 
moments,  i.e.,  E{|  yj[u)  |2}  <  +«.  The  mean-square  error  resulting  from  using  these 
functions  may  be  expressed  by  adding  and  subtracting  the  alleged  optimum 
functions  {p/(u)}.  The  conditional  moments  may  be  introduced  in  this  expression. 


=  Z  Ey  jEwy[|  a/y)  -  P/(y) + P/(y)  -  Y/(y)  f /y]| 


=  Z  Ey{Ey/y[|ay(u)-Py(u)|2/u  j  +  X  Ey{|P/(y)“Y/<y)f} 


The  value  e2,  as  identified  on  the  right,  is  a  minimum  because  of  the  following 
inequality  which  becomes  strict  when  Py(u)  and  yy(u)  differ  on  any  set  of  nonzero 
probability. 


n-k- 1 


I 


Ey||Py(y)-y/(y)|2}  s  0. 
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P(y)  =  [JVv/u(?  /  0)dz  +  uF  J  fy/y(z  /  O)dz]o. 


Thus,  in  terms  of  the  conditional  expectations,  the  optimum  estimator  contains  a  bias 
term  of  condition  moments  of  the  processing  noise. 


P(u)  =  Ev/y{z/0}+  uFQ.  (3.9) 

The  minimum  error  e2,  equation  (3-7),  may  be  written  incorporating  the 
consequences  of  additivity  properties.  One  part  of  this  expression  is  the  sum  of  the 
conditional  means. 


=  Eu{uFQQhFhyh}  +  Ei/u{?/0}QQl’FhEy{u1’} 
+  Eu{u}FQQhE!!/u{zh  /o}  +  E„,u{z  /  0)QQhE„,u{zh  /  0}  . 


(3-10) 


The  superscript  h  denotes  the  conjugate  transpose  of  the  vector  or  matrix.  Another 
important  part  of  e2  involves  the  sum  of  the  parity  values  written  as  a  bilinear  form. 

X  |a/(y)|2=  vQQhvh.  (3-11) 

y=0 

The  conditional  expectation  of  this  sum  may  be  simplified  by  using  the  same  change  of 
variables  as  above,  yielding  a  result  using  a  quadratic  form  of  the  vector  of  the  parity 
values,  a(y). 

Ewu{a(y)ah(y)/u}  =  Ey/yjzQQhzh/oJ  +  uFQQhEv/y{zh/o}  +  Ey/y{z/0}FQQhuh.  (3-12) 


Several  cancellations  occur  when  combining  this  equation  with  the  results  in 
equation  (3-10)  and  substituting  into  equation  (3-7)  for  e2.  In  particular,  it  is  interesting 
to  note  that  the  minimum  MSE  does  not  involve  any  averaging  over  the  input 
distribution. 
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e2  =  Ewu{^IQhj,,/s}-E»/y{z/2}QQhE!,/u{zh/Q}. 


(3-13) 


The  complete  description  of  the  hardware  errors  and  roundoff  and  quantization  noise 
is  contained  in  the  conditional  density  function  l^u(y/p).  It  may  have  a  mixed  type  of 
definition.  One  realistic  example  has  the  hardware  errors  following  a  Gaussian  density 
whereas  the  other  noise  can  have  a  uniform  distribution  determined  by  the  smallest 
level  in  the  numerical  representation  in  the  implementation  [16,19].  In  concrete  terms, 
an  example  of  a  modeling  density  function  ^u(£/£)  may  be  developed.  The  roundoff 
and  quantization  noise  may  be  lumped  together  and  modeled  as  k  independent 
components  in  a  vector  p  =  (fy,  . fy-i)* 


The  quantization  width  is  given  by  8  and  p_.,(x)  is  the  unit  step  function  [17].  On  the 
other  hand,  the  hardware  errors  obey  a  jointly  Gaussian  density  function  where  the 
covariance  matrix  A  and  mean  value  vector  p  permit  correlation  between  hardware 
error  events  and  bias  offsets  in  each  component  respectively  [20]. 


where 

and 


'a(i)  =  [(2K)k/z\A\y2^  ^xpj-^a-pjA  \a - p)T| 

3  =  (3q,  a. |, ....  ak_ i)  p  =  (pg,  p-j, ...,  p/f_i)  =  ((E{a^)) 

A  =  ((E{{aj-pj)  (aj-pj)}))  Covariance  Matrix. 


The  combined  effects  of  the  hardware  errors  and  performance  noise  is  included 
as  the  sum  of  the  random  vectors  a  and  p. 

£  =  a+b- 


Assuming  that  the  vectors  are  mutually  statistically  independent,  the  density  function  of 
p  is  the  /c-fold  convolution  of  the  two  density  functions  [20,25],  written  symbolically  as: 

's(c)  =  Jfs(c-x)f6(x)dx. 

X 
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This  density  function  is  related  to  the  modeling  conditional  density  through  the 
following  requirement: 

into®)  =  fcte)  • 

The  bias  term,  E^z/QJQ,  in  the  conditional  mean  estimator  j}(u),  equation  (3-9), 
does  not  usually  appear  in  the  formulation  of  the  parity  estimator,  it  has  generally 
been  ignored  in  the  past.  However,  omitting  it  can  have  an  impact  on  the  minimum 
mean-square  error.  When  this  offset  term  is  not  included  in  the  parity  estimations,  it  is 
easy  to  show  that  the  MSE  increases  by  an  amount  D, 

D  =  Eu{Es,/„{z/0}QQhE  v/y{?h  /0}}>0.  (3-14) 

Hence  the  parity-check  part,  Q,  of  the  generator  matrix  amplifies  the  effects  of  any 
noise  mean  E^lz/QJ. 

The  results  discussed  so  far  are  too  simple.  Hardware  errors  and  roundoff  and 
quantization  noise  can  appear  in  the  three  major  subassemblies.  A  more  realistic 
simulation  will  now  be  detailed.  A  common  assumption  in  fault-tolerance  practice 
constrains  only  one  subassembly  to  be  affected  by  hardware  failures  at  a  given  time. 
However,  roundoff  and  quantization  noise  occurs  in  all  three  parts  simultaneously. 
The  more  general  model  employed  below  will  incorporate  all  of  these  features.  The 
goal  is  to  determine  the  parity  estimation  functions  in  such  an  error  environment.  The 
previous  results  for  the  simpler  case  provide  important  guidance  in  including  the 
effects  of  hardware  error  and  performance  noise  in  the  estimation  subassembly  which 
has  not  been  fully  determined  yet;  a  classic  chicken-and-egg  problem.  The  key  steps 
in  this  regard  are  indicated  by  the  estimator's  equations  (3-9)  for  this  simple  situation, 
coupled  with  the  basic  interconnection  of  the  parts  shown  in  Figure  2-1 . 

Four  probability  events  will  be  employed  in  the  error  and  noise  model.  They  are 
categorically  listed  in  Table  3-1  along  with  the  symbolic  probabilities  of  their  respective 
occurrences.  Events  F,  E  and  C  represent  both  hardware  errors  and  .  oundoff  and 
quantization  noise  effects,  whereas  event  N  accounts  for  the  simultaneous  roundoff 
and  quantization  noise  in  the  three  subassemblies.  However,  the  generalized  additive 
properties  of  the  conditional  probability  density  functions  describing  all  forms  of  error 
suggest  a  realistic  tractable  modeling  technique.  The  noise  offsets  in  the  parity 
estimation  part  can  be  reflected  through  the  comparator  and  consideied  an  additional 
contribution  in  the  parity  construction  part.  The  linear  nature  of  the  simple  parity 
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estimators,  equation  (3-9),  coupled  with  the  adder/subtractor  in  the  comparator  slices, 
Figure  3-1  a,  indicate  that  extra  components  of  noise  added  to  the  parity  construction 
process  accurately  model  the  aggregate  effects  from  the  estimation  subassembly.  The 
exact  form  of  the  optimum  MSE  parity  estimation  function  vector,  £(11),  will  be 
developed  using  the  model.  Note  the  model  has  roundoff  and  quantization  effects 
active  at  all  times. 

Table  3-1.  Error  and  Noise  Events  for  Subassemblies 


Event 

Subassembly 

Nature  of  Errors 

Probabilities 

F 

Linear  Signal  Processing 

Soft  and  Numerical 

Pf 

C 

Parity  Construction 

Soft  and  Numerical 

Pc 

E 

Parity  Estimation 

Soft  and  Numerical 

Pe 

N 

Processing,  Construction  and 
Estimation 

Numerical  only 

Pn 

1  =  Pp  +  Pc  +  Pe  +  Pn 
Pn  >:>  Pf  +  Pc  +  Pe 

The  previous  development  of  the  mean-square  error  estimators  in  equations 
(3-6)  and  (3-7)  is  the  starting  point  in  the  case  of  the  above  expanded  model.  It  is  easy 
to  show  as  earlier  that  the  optimum  estimator  functions  in  vector  form  may  be  written 
using  the  events  and  probabilities  defined  in  Table  3-1. 

P(u)  =  pp  E{a(v)|F,u}  +  pc  E{a(v)|C,u}  +  pE  E{a(v)lE,u}  +  pm  E{a(v)|N,u}.  (3-1 5) 

Each  conditional  expectation  will  be  determined  individually.  The  optimum  mean- 
square  error  expression  may  be  developed  as  before  and  can  be  compactly 
expressed  using  vector  notation: 

e2  =  Ey  |e[  a(y)ah(v)/u  ]  -  g(u)|h(u)|.  (3.1 6) 

A  joint  density  function  involving  u.  y.  and  &(¥.)  will  be  useful.  It  may  be 
decomposed  according  to  the  four  events  detailed  in  Table  3-1  and  contains 
appropriate  conditional  density  functions. 
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f(y.¥.a)  =  yufrF^v-aF.y)  +  Pc^(¥.a|C,u)  +  pEf(v,a|E,u)  +  pN%a|N,u)]  (3.1 7) 


It  must  be  remembered  that  the  processing  output  data  in  y  and  its  associated  parity 
tt(y)  may  have  a  statistical  relationship  in  addition  to  their  deterministic  one  defined 
through  parity  equations  (3-11).  Hardware  errors  and  roundoff  and  quantization  noise 
in  the  parity  calculation  subassembly  introduce  this  extra  dimension. 

The  first  part  of  the  estimator  p(u)  depending  on  event  F,  where  the  errors  are 
introduced  in  the  signal  processing  part,  can  be  developed  as  before,  leading  to  a 
result  similar  to  equation  (3-9). 

E{a,v|F,u}  =  Ey/Fiy{?|F  ,0}  +  uFQ.  (3-1  q) 

On  the  other  hand,  event  C  corresponds  to  hardware  errors  and  performance  noise 
appearing  in  the  parity  construction  part.  In  that  case,  the  pertinent  conditional  density 
function  has  a  singular  component  allowing  the  following  constraint: 

f(v,a|C,u)  =  ^|C,y.y(a|C,u,y)fv|c,u(v|C,u)  (3.19) 

=  'g|C,u,y(a|C,u,y=uF) 

=  'a|C.v(8|C,!£)|sssuF 


Furthermore,  the  last  conditional  density  function  has  a  generalized  additive  property. 

'slC.yfelC.y)  =  fg|C,y(a  “  vQ|C,0).  (3.20) 

The  component  of  p(u)  related  to  event  C  may  be  evaluated  with  the  aid  of  a  change  of 
variables,  a -yQ. 


E{a(v)|C,u}  =  Ea,Ciy{y|C,0}  +  uFQ.  (3.2i ) 

The  influence  of  errors  in  the  parity  estimation  subassembly,  event  E,  is  handled 
the  same  way  except  that  the  conditional  density  function  modeling  errors  in  the  p(u) 
calculation  is  transferred  to  the  parity  construction  part.  The  resulting  conditional 
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expectation  involves  an  a(y)  item,  but  the  underlying  density  function  E  |  E,y) 
represents  the  errors  in  the  estimation  part. 

E{a(v)|E,u}  =  Ea|E,ytelE,Q}  +  yFQ  (3-22) 

For  the  case  of  event  N  where  roundoff  and  quantization  effects  occur  in  all 
three  subassemblies,  the  influence  in  the  parity  estimation  part  is  modeled  by  adding 
appropriate  statistical  contributions  in  the  parity  construction  subassembly.  Then  the 
conditional  density  function  describing  the  filtering  and  parity  construction  parts  may 
be  separated  using  Bayes’  Rule  [20]. 

f(M.a|N,u)  =  ^|N,u(v|N,u)/a|N)yiv(a|N,u,v).  (3_23) 

The  first  conditional  density  on  the  right  represents  noise  in  the  processing  part 
whereas  the  second  item  represents  the  combined  noises  in  the  parity  construction 
subassembly.  The  nature  of  the  signal  flows  shows  that  the  last  conditional  density  is 
not  dependent  on  the  input  data  in  u- 

's|N.y,v(a|N,u,v)  =  fS|Niv(a|N,v).  (3.24) 

Furthermore,  two  generalized  additivity  properties  come  into  play: 


fY|Hu(v|N,u)  =  fy_  |N,y(y  -  uF|N,0)  (3_25a) 

fg|N,y(a|Hy)  =  fg|N,v(«  “  vQ|N,0)  (3-25b) 

The  conditional  expectation  of  interest  involves  a  double  integral  and  two  changes  of 
variables,  given  by  *  ■  y  -  uF  and  y  =  a  -  yQ,  produce  three  parts  to  the  conditional 
expectation, 

E{a(y)|N,u}  =  Es)Ni  y{y  |N,0}  +  Ev!N,y{z|N.Q}Q  +  uFQ.  (3.26) 

The  complete  expression  for  j3(y)  combines  items  from  equations  (3-18),  (3-21), 
(3-22)  and  (3-26).  The  term  uFQ  appears  without  any  event  probability  weighting 
because  Pf  +  Pc  +  Pe  +  Pn  ■  1  •  A  complete  bias  term  which  does  not  depend  on  input 
data  u.  is  given  by  a  vector  £. 
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where 


P(y)  =  uFQ  +  C, 


(3-27) 


E  =  PFEv|F>y{2lF.Q}Q  +  PCEa|C,v{ylC-Q}  +  PEEa|F,v{?lE.Q} 
+  PNEfi|N.y{y|N,Q}  +  Pnev!N,u{z|N,Q}Q 


(3-28) 


The  1  x  (n-  k)  bias  vector  £  accounts  for  any  nonzero  noise  averages  both  due  to 
hardware  errors  and  roundoff  and  quantization  considerations.  This  vector  will  appear 
in  the  minimum  MSE  e2. 

The  other  contributing  factor  to  the  MSE,  E{&(y)  ah(y)  I U},  may  be  expanded  in 
a  sum  of  conditional  expectations  according  to  the  four  events  detailed  in  Table  3-1 . 
These  individual  conditional  expectations  may  be  developed  similar  to  ones  above, 
except  that  the  arguments  involve  an  inner  product  type  expression  &(y)  ah(y). 
However,  the  manipulations  and  simplifications  applied  earlier  are  still  pertinent.  This 
development  is  straightforward  but  tedious.  The  final  minimum  MSE  may  be 
assembled  according  to  equation  (3-16)  and  careful  bookkeeping  shows  that  all  terms 
containing  uoruh  cancel.  Many  of  the  remaining  items  contain  correlations  between 
noise  variables. 

e2  =  pFEv|F,y{zQQhZh|F,U  =  o}  +  PcEa|C,v{yyhiC>¥  =  o}  +  PEEa|E,y{xxh|E.  V  =  o} 

+  Pn  Ea|N.v{yyh|N, V  =  o}  +  pN  Ea|N,v{y  i Ky  =  0}Qh  Ey|N,u{zh|N,y  =  o}  (3-29) 

+ Pn  Ey|N,y{2  |N.y = 0}Q  ES|N,y{yhIH  y  =  o}  +  pn  Ey|NiU{zQQhzh|N,u  =  o}  -  rrh. 

The  results  for  the  noise  model  used  here  indicate  the  form  of  g(u),  equation 
(3-27),  would  introduce  the  same  type  of  noise  as  the  parity  construction  subassembly. 
This  justifies  the  reflection  of  the  noise  contribution  from  the  estimation  part  to  the 
construction  subassembly. 

Infinite  Input  Data  Stream 

The  case  where  the  linear  signal  processing  handles  an  infinite  input  data 
stream  will  be  protected  through  parities  from  a  real  convolutional  code.  The  general 
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situation  will  be  narrowed  to  a  more  practically  important  setting.  Nevertheless,  it  will 
be  obvious  from  the  development  that  more  general  results,  similar  to  those  for  the 
finite  length  processing  case,  can  be  determined  in  a  straightforward  manner.  The  first 
of  two  constraints  assumes  the  signal  processor  is  an  HR  filter  with  a  rational  Z 
transform.  The  corresponding  matrix  F  has  the  infinite  impulse  response  [hr}+“0  as 
columns. 


fy)  *•*  hr 

0  /Jq  •••  /)/■_  i 

0  0  : 

0  0  : 


F  = 


fh 

ho 

0 

0 


..A 


HR  filter  matrix 


V 


J 


(3-30) 


Most  practical  weighting  systems  are  based  on  such  types  of  designs  [16-19]. 

The  other  simplifying  constraint  concerns  the  real  convolutional  code  that 
determines  the  parity  values.  A  high  rate  code  with  only  one  parity  channel  will  be 
employed.  This  means  that  the  information  positions  are  grouped  by  k  =  n  -  1 
samples,  giving  a  code  rate  of  [(n-  1)/n].  It  is  possible  to  relax  this  constraint  providing 
more  code  alternatives,  but  for  the  moment  this  approach  is  notationally  convenient. 
The  corresponding  systematic  real  convolutional  code,  described  earlier  in  equations 
(2-6)  through  (2-8),  employs  a  parity  channel  weighting  matrix  Q  with  a  simplified 
notation. 


Q  = 


;  M  =  (m  + 1)(/7  - 1). 


(3-31) 


The  parity  construction  part  of  the  system  produces  samples  for  this  one  parity  channel 
according  to  the  following  formula: 
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;  r*>  0. 


(3-32) 


a(v«)  =  S' 

/= 0 

The  truncated  output  data  stream  denoted  by  yM  is  given  notationally  as 

y(/)  =  (v0,  Vi . v,)  ;  rt  0.  (3-33) 

Only  the  last  M  samples  in  this  data  vector  affect  the  parity  construction,  but  the 
corresponding  input  data  samples  from  u  contributing  to  yM  are  represented  by  a 
similar  truncated  vector. 


. Uf)  ;  rz  0.  (3-34) 

The  outputs  from  the  signal  processing  operations  arise  from  the  usual 
convolution  expression. 


t  t 

Vt  =  X  hiut-i  =  X  ht-juj  ;  t  = 
i= 0  /=0 


(3-35) 


On  the  other  hand,  the  parity  estimation  subassembly  provides  the  parity  stream 
denoted  by  jpf(uW)J^  .  It  minimizes  the  mean-square  error  between  corresponding 
parity  values.  The  MSE  is  denoted  by  ef  where  t = 0, 1, ... 


4  =  E 


«f(v(f))_pf(y(0)|2 


(3-36) 


It  may  be  shown  as  before  that  the  estimator  minimizing  this  symbol-by-symbol 
criterion  is  the  conditional  mean  of  the  parity  construction  variable  a^yM). 

Pf(y(f))  =  E{af(y<f))/uW}  (3.37) 

The  proof  supporting  this  and  the  development  of  the  related  mean-square  error,  if, 
follows  the  techniques  used  earlier. 
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(3-38) 


When  the  only  source  of  errors,  both  hardware  and  roundoff  and  quantization 
effects,  is  the  signal  processing  subassembly,  the  parity  estimator  can  be  developed 
paralleling  the  previous  analysis.  The  errors  are  described  by  a  conditional  density 
function  which  is  assumed  to  obey  a  familiar  generalized  additivity  property.  The 
optimum  estimator  has  the  following  form. 


P^°)  =  +  sV*  I  (3-39) 

/= 0  “  /= 0  j=Q 

This  estimator  contains  the  parity  of  the  filtered  input  as  does  an  expected  value 
needed  in  e*. 


=“x  1 

/= 0  j= 0  1  ■ 


(3-40) 


Not  surprisingly,  the  minimum  MSE  is  only  dependent  on  the  finite  length  of 
data  needed  in  the  parity  construction  subassembly  even  though  samples  for  the 
entire  length  of  uto  and  yto  appear  in  the  two  previous  expressions. 


M- 1 

*?=  I 

hO 
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The  correlation  between  noise  samples  in  the  processing  subassembly  plays  a 
prominent  role  just  as  in  the  finite  length  processing  case,  e.g.,  equation  (3-13). 

The  situation  where  hardware  errors  are  permitted  in  any  one  of  the  three 
subassemblies  while  roundoff  and  quantization  disturbances  are  present  in  all  parts 
can  be  addressed  employing  the  same  error  model  described  in  detail  earlier.  The 
nature  and  type  of  error  events  remain  as  given  in  Table  3-1.  A  set  of  familiar  results 
emerges.  The  parity  estimation  part  contains  a  bias  term  in  addition  to  the  expected 
parity  affiliated  with  the  processing  part’s  output. 


/= 0  j-0 

where 

M= 1  M= 1 

At  =  PF  X  ^^vW/F  +  pN  X  ^  '  Ev(0/Nu('){Z'-'/N’Q} 

/= 0  “  ’■  /= 0 

(3-42b) 

+  PN  Ea(f)/N,  5/(0^^-}  +  Pc  Ea(^/C,y(f){y//C,-i  +  pEEa(#)/E,y(0^/E’QJ 
and 

M=1  M=\  ...  .  ,.t  /  *  /  ^ 

=  pN  X  X^'V7*  ev(0/n.u(o{^-'  zt-j/N’Q} 

i- 0  /= 0 

+  PF  X  iV'V"*  Ev(0/Fu(0{z,.fz^,0} 
i-0  7=0 

+  PC  Eo<')/C.v('){lJ,'|2/C’S}  +  PE  Eo(0/Cl,(0{l«^2}  +  PN  Ea<')/N,v('){ly'|2^,Q}  <3‘43> 

+  PN  I  <7(0  E2(0/n.u('>K'/M 

+  pn  Z1  e0(')/n,v(o{^2}  -  lA<f- 


3-16 


It  is  easy  to  identify  the  sources  of  the  noise  effects  in  both  the  estimators  and 
the  minimum  MSE  value  ef.  Those  items  containing  conditional  expectations  with 
subscripts  of  the  form  yM/A.yM  (where  A  is  a  generic  letter)  represent  noises  in  the 
signal  processing  part.  On  the  other  hands,  those  conditional  expectations  having 
subscripts  of  the  form  a^/A.yW  designate  noise  sources  in  either  the  parity  calculation 
part  or  parity  estimation  subassembly  as  reflected  through  to  the  construction  part. 
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IV.  EFFICIENT  USE  OF  REAL  CODES 


The  optimum  mean-square  error  parity  estimator,  whether  for  finite  or  infinite 
length  signal  processing  operations,  contains  a  term  representing  the  parity  of  the 
processed  output.  For  finite  length  data  processing,  the  term  uFQ  contains  the 
processed  output  uF  (equations  (3-9)  or  (3-27)).  On  the  other  hand,  when  processing 
a  continuous  infinite  input  data  stream  the  double  sum 

t-i 

7=0  J 

is  the  parity  for  M  outputs  of  the  processing  subassembly  (see  equations  (3-37)  or 
(3-42)).  At  first  glance  it  appears  that  the  parity  estimator  must  duplicate  the  complete 
processing  effort  of  the  signal  processing  subassembly,  implying  that  the  protection 
overhead  must  exceed  100%.  Thus  the  efficiency  of  the  parity  estimator  subassembly 
is  an  important  factor  in  offering  any  practical  advantages.  Fortunately,  for  both  finite 
and  infinite  length  processing  systems,  techniques  exist  for  dramatically  reducing  the 
computational  requirements  in  the  parity  construction  and  estimation  subassemblies. 

An  effective  way  to  reduce  parity  computation  complexities  is  to  match  the 
processing  calculation  more  closely  to  the  structure  of  the  real  code.  Two  techniques 
will  be  discussed.  In  the  first  approach,  close  similarity  between  finite  convolution  and 
real  cyclic  codes  reduces  the  parity  estimation  calculations  to  the  order  of  the  number 
of  parity  values  squared.  On  the  other  hand,  the  parity  channels  used  to  protect  infinite 
impulse  response  filtering  can  be  modified  so  as  to  cancel  the  poles  of  the  original 
transfer  function.  This  not  only  removes  the  need  for  feedback  memory  requirements 
in  the  parity  estimation  part,  but  more  importantly,  permits  the  parity  construction  and 
estimation  subassemblies  to  operate  at  lower  computational  rate.  This  in  turn  offers 
very  efficient  implementation  options.  The  details  of  both  approaches  will  be  given 
below,  starting  with  protecting  finite  length  convolutions. 

Protection  With  Real  Cyclic  Codes 

Finite  length  convolutions  may  be  described  compactly  in  terms  of  polynomials 
[26].  The  filter  weighting  sequence  may  be  viewed  as  a  polynomial  f(X)  whose 
coefficients  are  related  to  the  signal  processing  matrix  in  the  following  way. 


m-i  ... 
/= 0 
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kxk  Convolution  Matrix,  s  £  k 


(4-1  a) 


^0  U  * 2 

0  f0  f, 

o  0  f0 

:  0  0 


0  0  0 


U 

fo 

0 


/(X)  =  +  f^X8"1  +  ...  +  +  /0  ;  Weighting  Polynomial  (4-1  b) 


When  the  input  data  are  represented  in  polynomial  form,  u{X),  the  convolution 
between  the  input  data  and  the  weighting  sequence,  /(X),  is  a  polynomial  product 
f(X)  u(X).  If  the  integer  k  is  chosen  large  enough  (deg  f{X)  +  deg  u(X)  <  k),  the 
convolution  weighting  may  be  viewed  in  a  residue  polynomial  ring  modulo  (X*-  1) 
[24,26]  where  the  output  convolution  is  contained  in  polynomial  v(X), 

v(X)  =  u(X)  f(X)  modulo  (X*-  1) .  (4-2) 


Real  cyclic  codes  are  defined  in  a  similar  residue  polynomial  ring  where  the 
modulo  is  performed  (XP- 1),  n>  k.  The  code  has  (n-  k)  parity  positions  and  length  n. 
It  is  defined  by  a  single  polynomial,  called  the  generator  polynomial  and  designated 
here  as  g(X).  This  polynomial  has  degree  ( n  -  k),  may  be  selected  with  a  unity  leading 
coefficient,  and  divides  (X”-  1).  The  code  is  all  the  polynomials  in  a  principal  ideal 
[24]  generated  by  g(X).  This  ideal  is  denoted  by  double  parentheses  around  g(X),  (( 
g(X) )),  and  is  formally  defined  as  the  set  of  polynomials  in  the  residue  polynomial  ring 
that  are  multiples  of  g{X): 

(( 0(X) ))  »  WX)  a  q(X)  g(X)  modulo  (X^1):  q(X)  a  polynomial}  (4-3) 

A  common  method  for  defining  a  systematic  form  of  a  cyclic  code  employs  the 
famous  Euclidean  algorithm  [24,26]  which  will  be  stated  here  for  completeness.  Any 
polynomial  f{X)  may  be  divided  by  g(X)  and  expressed  uniquely  in  terms  of  a  quotient 
q(X)  and  a  remainder  r(X),  with  an  important  constraint  on  the  remainder’s  degree: 
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W  =  q(X)  g(X)  +  r(X) 


deg  r{X)  <  deg  g{X) . 


(4-4) 


Cyclic  codes  over  the  real  or  complex  fields  are  defined  by  using  consecutively 
indexed  powers  of  the  /7th  complex  root  of  unity,  e.g.,  [exp(J2n/n)p],  where  j f  ■  V- 7" and  p 
is  an  integer.  The  fundamental  construction  techniques  are  given  by  Marshall  [11,27], 
and  use  the  discrete  Fourier  transform  domain  in  which  contiguously  indexed 
transform  coefficients  determine  the  generator  polynomial.  By  requiring  conjugate 
roots  be  included,  real  generator  polynomials  are  constructed.  Maximum  distance 
separable  codes  [11,13]  (analogous  to  powerful  Reed-Solomon  codes)  are  easily 
constructed.  It  was  established  in  [27]  that  real  number  maximum  distance  separable 
codes  exist  for  all  choices  of  parameters.  Such  codes  can  detect  errors  equal  in 
number  to  the  degree  of  g(X),  the  maximum  permitted  by  the  Singleton  bound  [13]. 

Real  cyclic  codes,  primarily  applicable  to  floating  point  arithmetic  formats,  will 
be  examined  in  slightly  more  detail  so  as  to  better  exemplify  the  parity  operations  to  be 
discussed  later.  They  are  constructed  using  consecutively  indexed  primitive  roots  of 
unity  [11]  and  were  originally  called  Discrete  Fourier  Transform  (DFT)  codes 
[11,12,27,28].  Powers  of  the  nth  complex  root  of  unity,  W,  define  the  roots  of  the  code 
generator  polynomial  g{X). 

gm=n.(x-wt):  (4.5) 

S  =  INDEX  SET  OF  CONSECUTIVE  INTEGERS  MOD  n 
S  =  [0,1,2 . (n-1)} 

The  span  (number  of  consecutive  indices)  determines  the  error-detecting  capability  of 
the  code  and  is  the  maximum  allowable  for  a  linear  code.  The  code  can  detect  up  to 
|  S  |  symbol  positions  in  error  considering  the  roundoff  tolerance  of  the  comparison 
operation. 

If  the  index  root  set  is  symmetric  about  0  (  and  including  0)  or  about  n/2  (and 
including  n/2  if  n  is  even),  conjugate  root  pairs  appear  in  g(X),  giving  it  real 
coefficients.  This  restriction  narrows  the  range  of  parameters  permitted  in  the  code 
slightly  (see  Property  3  of  [1 1]),  but  does  not  reduce  the  error  protection  levels  in  any 
way.  In  addition,  the  real  coefficients  of  g{X)  are  also  symmetric  or  anti-symmetric 
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about  the  degree  midpoint,  halving  the  number  of  multiplicative  operations  needed  in 
parity  calculation  and  re-calculation.  This  implementation  detail  will  not  be  exploited 
further  in  the  paper  to  avoid  unnecessarily  complicating  the  equations. 

A  power  of  the  primitive  complex  root,  Wm,  where  m  is  relatively  prime  to  n  can 
be  used  in  place  of  W  in  the  definition  of  g(X).  A  different  code  results  with  the  same 
error-detecting  capability  but  with  roots  located  at  more  widely  dispersed  points  on  the 
unit  circle.  This  eases  the  accuracy  requirement  in  calculating  and  using  the 
coefficients  in  g(X).  A  simple  example  for  a  code  with  length  n  =  1024  and  information 
capacity  k  =  1003  using  an  index  scaling  factor  m  =  47,  has  the  capability  of  detecting 
up  to  21  positions  in  error  or  any  burst  up  to  length  21.  Figure  4-1  displays  the 
generator  polynomial.  An  equally  good  code  may  have  roots  centered  about 
-1  =  W612,  but  they  have  different  coefficient  signs  since  (X+1)  is  the  only  linear  factor. 


g(X)  =  -1.000000e+00  +7.8827336-01X1  -7.0639406-01X2 

+6.591 348e-0 IX3  -6.277460e-01X4  +6.055386e-01X6 
-5.894287e-01  X6  +5.777794e-01  X7  -5.69651 4e-01X® 
+5.644898e-01  X9  -5.6198066-01X10  +5.6198066-01X11 
-5.6448986-01X12  +5.69651 46-01X1 3  -5.7777946-01X14 
+5.894287e-01  X1 5  -6.0553866-01X16  +6.2774606-01X17 
-6.591 3486-01X18  +7.0639406-01X19  -7.8827336-OIX20 
+1.0000006+00X21 

Generator  Polynomial  Using  Root  W47 

n=  1024  ;  fc=  1003 

A  REAL  CYCLIC  CODE  GENERATOR  POLYNOMIAL 

FIGURE  4-1 

A  common  systematic  encoding  scheme  will  be  used  to  describe  the  efficient 
calculation  of  parity  values  associated  with  the  output  convolution.  The  input  data 
represented  by  u(X)  is  encoded  in  the  following  format.  The  data  in  u(X)  are  placed  in 
higher  coefficient  positions  by  multiplying  it  by  X'*'**,  effectively  shifting  this  data  to 
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inclusively  indexed  positions  from  (n-  k)  up  to  [n  -  1).  The  uniquely  related  parity 
symbols  are  represented  by  the  polynomial  ru(X),  derived  from  the  Euclidean 
algorithm  (4-4)  with  g(X)  as  the  divisor. 

XP~ku{X)  =  qu(X)  g(X)  +  ru(X)  ;  deg  ru(X)  <  deg  g(X)  (4-6) 

ENCODE  u{X)  ->  XP~ku{X)-ru{X) 


PARITY  PART  =  -rj(X) 

The  parity  estimation  subassembly  may  use  the  parity  ru(X),  which  is  carried 
along  with  the  input  data  u(X),  in  forming  the  parity  -rJiX)  corresponding  to  the  output 
v(X),  equation  (4-2).  The  formula  relating  the  input  parity  - ru(X)  to  the  output  parity 
r^X)  involves  a  weighting  polynomial  \y(X)  which  may  be  computed  and  stored. 
Because  v(X)  is  the  remainder  modulo  g(X),  it  has  degree  strictly  less  than  ( n  -  k). 

r^X)  s  ru(X)  \y(X)  modulo  g(X)  (4-7a) 

where 

\jr(X)  &  /(X)  modulo  g(X) .  (4-7b) 
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The  proof2  that  this  is  indeed  the  proper  output  parity  is  contained  in  a  footnote  which 
makes  use  of  the  properties  from  the  Euclidean  algorithm.  The  computation  of  the 
parity  estimator  uses  equation  (4-7a)  which  is  much  simpler  than  a  straightforward 
computation.  Its  complexity  is  on  the  order  of  (n  -  k)2.  Furthermore,  if  the  parity  values 
may  be  computed  and  compared  in  a  suitable  transform  domain,  this  complexity  drops 
to  the  order  of  ( n  -  k)  [29]. 

The  systematic  generator  matrix  for  a  cyclic  code  may  be  described  by  mixing 
vector  and  polynomial  notation.  The  /*h  row  of  G  relates  to  the  cyclic  codeword  for 
{X/K1“#}  where  r=  0, 1, ....  k-  1.  In  particular,  when  this  code  polynomial  denoted  by 
{XrM_r- p,lX)}  is  written  as  a  vector,  the  Xn-1“r  terms  correspond  to  the  identity  part  lk 
while  the  parity  parity,  remainder  -  p/X),  is  confined  to  the  rightmost  (n-k-  1)  matrix 
columns. 

Xn-1“r  =  q^X) g(X)  +  pJ(X)  ;  deg  p^X)  <  n-k  (4-8) 

2  PROOF  OF  SIMPLIFIED  PARITY  CALCULATION,  EQUATION  (4-7). 

The  parity  associated  with  the  output  v(X)  =  u(X)  f(X)  modulo  ( Xk -  1)  will  be 
denoted  by  £(X).  The  Euclidean  algonthm  shows  there  is  a  quotient  q/X)  and  a 

remainder  ¥(X)  obeying 

i{X)  =  q^X)  g(X)  +  ¥(X) . 

This  gives  equivalence  (4-7b).  On  the  other  hand,  the  product  ru(X)  ¥(X)  when 
reduced  modulo  g(X)  has  remainder  denoted  by  rj(X). 

ru(X)  V(X)  =  qp(X)  g(X)  +  rJ(X) . 

The  validity  of  equation  (4-7a)  may  be  seen  by  expanding  the  difference  between 
the  shifted  output  data,  X^t^X)  /(X)],  and  the  parity  rJ(X). 

Xn~k[u(X)  VO]  -  o  -  {q JX)g(X)  +  rJX))  {q^X)g(A)  +  MX)}  -  <fe«X)0(JO  ♦  ^•’OvC’O 

=  0  +  %(mx)vW  -  qJ.X)gW  ■ 

Since  rJ(X)  has  degree  less  than  (n-k)  and  is  unique  by  its  very  construction,  it 
must  be  the  corresponding  parity  because  the  difference  above  is  a  multiple  of  the 
code  generator  polynomial  g(X). 
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In  order  to  be  somewhat  consistent  with  the  earlier  block  code  notation  associated  with 
equation  (2-4),  the  coefficients  have  a  reversed  order. 

PrW  =  ZPr./  I  P.  r=0.  t ....  (k-1).  (4-9) 

/=0 

Then  the  systematic  generator  matrix  G  in  the  cyclic  code  has  the  following  form 
involving  matrix  P  defined  by  remainder  polynomials  p^X). 

G  -  (V I  P)  =  HXT-'-'-p/tm  ;  r=0,1 . (A-  1) .  (4-10) 

The  parity  construction  subassembly  implements  the  following  function  where 
the  reverse  indexing  and  minus  sign  appear  because  of  the  notational  definition  of  the 
real  cyclic  code. 


k- 1 

<*r(v)  =  -  5>/p*-w  •  r=0*  t  ••••  (* -1)-  (4.11) 

This  definition  gives  parity  values  as  generated  by  the  Euclidean  algorithm  encoding 
format,  equation  (4-6).  The  encoding  of  output  v{X)  has  a  parity  polynomial  part  r^X) 
that  may  be  expressed  using  the  a,&)  values. 

Xn~ki/(X)  .  qJ.X)q(X)  *  rJX)  (4-1 2a) 


rv(X)  =  - 


n-k-1 

Z  a/(v)X' 


L  i*0 


(4-1 2b) 


Typical  terms  in  the  optimum  MSE  value  e2  contain  bilinear  forms  of  the  parity 
matrix  Q  and  other  statistical  quantities,  e.g.,  equations  (3-13)  or  (3-29).  Some 
simplifying  assumptions  will  be  made  in  order  to  study  the  impact  on  MSE 
performance  by  choices  for  the  parity  coefficients  in  parity  part  matrix  P,  equation  (4-9). 
These  parity  coefficients  depend  on  the  generator  polynomial  which  in  turn  is  defined 
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by  its  indexing  subject  E,  equation  (4-5).  This  will  be  explored  further  by  concentrating 
on  one  bilinear  form,  say  item  E. 

E  =Ev/y{zQQhzh/o}  =  X  S  X  Pfc-1-i.r  P#c-1-y.r  Ey/y  [^i  ^}/o]  (4-13) 

r= 0  i—0  /= 0 


This  bilinear  form  contains  correlation  statistics  concerned  with  errors 
appearing  in  a  subassembly’s  error  model.  As  a  beginning  point,  the  noise  will  be 
taken  as  uncorrelated  so  that  individual  component  noise  powers,  ay,  are  the  only 
nonzero  statistics. 


The  bilinear  form  reduces  to  a  sum  of  squares. 


n-k- 1  k- 1  9 

E=  I 

r=Q  i=0 


(4-15) 


Note  that  the  noise  powers  oy  are  not  necessarily  from  a  wide-sense  stationary 
process  [20].  It  is  easy  to  show  that  such  a  situation  arises  in  a  direct  realization  of 
finite  convolution  when  scalers  and  summers  are  affected  by  uncorrelated  noise.  Also 
the  oy  have  a  linear  relationship  for  indices  in  the  beginning  and  end  of  the  indexing 
range. 

Real  cyclic  codes  are  defined  using  linear  complex  roots  of  unity,  and  it  is 
natural  to  introduce  a  transform  in  E.  It  appears  that  a  Parseval’s  type  relationship 
might  apply  to  the  squared  items  in  equation  (4-15).  Therefore,  some  notational 
conventions  will  be  introduced  for  a  discrete  Fourier  transform  [16-19,26-28].  The  data 

sequence  will  be  denoted  by  {a,},  r  =  0, 1 . (n  - 1 ),  and  its  transform  is  designated  by 

the  transform  sequence  {A^},  q = 0,  \ ....  (n- 1). 


n- 1 

;  q  =  o,\ 


r=0 


(n-1). 


(4-1 6a) 


The  usual  inverse  transform  applies: 
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(4-1 6b) 


1  n-l 

ar  =  -  X  A^W"^  ;  r  =  0,  L  ....  (n  -  1) 
n  <7=0 

The  transform  pair  will  be  used  on  individual  rows  of  the  generator  matrix  G, 
equation  (4-10).  The  /'th  row,  n  samples  long  produces  a  transform  sequence  {P/.g}- 

P,.,  =  ; 

r= 0 

(4-17) 

q  =  0,  \  ....  (n  -  1)  ;  i  =  0,  1 . (fc  -  1) . 

The  first  item  is  from  the  monomial  X^1-'  representing  the  identity  part  of  G  while  the 
remaining  sum  arises  from  the  parity  poiynomial-p,<X)  for  this  /th  row. 

The  cyclic  code’s  definition  designates  which  powers  of  W  are  roots  of  the  rows 
of  G,  viewed  as  polynomials.  From  equation  (4-5),  if  index  £  e  2,  the  index  subset,  W^, 
is  a  root  of  each  [X^-'  -  pjX)].  This  gives  an  evaluation  of  p,(X)  at  all  (n  -  k)  indices 
in  2. 

p,(w^  =  ^ e2  ;  /  =  0,  X  ....  (k  -  1) .  (4-18) 

On  the  other  hand,  the  Lagrange  interpolation  formula  [16,26]  shows  how  each  p,<X), 
degree  (/7  —  Ac  — 1 )  or  less,  is  completely  specified  by  knowing  the  ( n-k )  values  given 
by  equation  (4-18). 


PAX)  =  I 

This  interpolation  result,  when  combined  with  equation  (4-1 8),  yields  a  closed  form 
expression  for  the  transform  of  the  rows  of  G. 


n(*-*n 

_ 

YlW^-W*  (4-19) 

Jx.aeS 
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(4-20) 


piq  =  wq(n-i~i'>- 


_ 

Y[W%-W° 


IX,0€i 


The  roie  of  the  root  indexing  subset  E  is  obvious  now. 

The  inverse  transform  may  be  invoked  to  establish  the  following  ParsevaPs  type 
relationship  expressing  certain  squared  terms  in  E,  equation  (4-15). 


P/c-W.r  f  =  4r  I  I 
n  t=0  q=0 


(4-21) 


Then,  performing  the  summation  in  equation  (4-15)  over  index  r  leads  to  a  result 
where  further  simplifications  are  possible  because  of  the  nonoverlappirg  nature  of 
some  summation  indices. 


where 


E  = 


X  X 


7=0  i=0 


X  X  w'^coO.^o)  (y.c) 


(4-22a) 


<a(7,y  =  -  X 

n  t=Q 


^ _ 


lX.ceZ 


(4-22b) 


The  noise  power  terms  of  are  fixed  by  the  noise  statistics  and  are  not  affected 
by  the  code  indexing  subset  S.  The  summation  index  /  in  (4-22a)  on  some  terms 

resembles  a  transform  of  the  length  k  sequence  of  j,  /  -  0, 1 . (k  -  1)..  The 

following  is  a  legitimate  transform  sequence. 


*-i 

S (r)  =  £  of  tV/r  ;  r  -  0,  \  ....  (n  -  1) . 
/= 0 


(4-23) 
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Thus,  the  bilinear  form  E  can  be  rewritten  compactly. 


I  Sft-0  PsftC) 

with  the  identification, 

<««)  = 

y=o 

The  function  Q  has  (n  -  /c)2  values  and  there  are  n  starting  positions  for  the 
indexing  subset  S.  Thus  it  is  possible  to  find  the  best  indexing  subset  by 
straightforward  calculations  on  the  order  of  n(n  -  k)2. 

Modifying  Real  Convolutional  Codes 

Infinite  sequence  processing  described  by  a  finite  difference  equation  (or 
equivalently,  by  a  rational  Z  transform  transfer  function)  will  be  protected  by  a 
systematic  real  convolutional  code.  The  key  to  simplification  lies  in  adjusting  the  parity 
calculation  equations  so  as  to  cancel  the  poles  of  the  transfer  function,  permitting 
computation  in  the  parity  estimation  subassembly  at  a  slower  rate.  The  details  of  this 
novel  approach  follow. 

The  finite  difference  equation  relating  input  to  output  samples  is  defined  by  two 
sets  of  coefficients. 


(4-24a) 


(4-24b) 


v  6 

vr  =  Xa/Wr-/  “  'Lbivr-i  •  (4-25) 

/= o  y=i 

The  corresponding  transfer  function  H(Z)  can  also  be  viewed  as  the  ratio  of  the 
respective  Z  transforms  of  the  input  and  output  sequences.  It  is  completely  defined  by 
the  coefficients  governing  the  difference  equation. 

INPUT  { u f}  U{Z) 

OUTPUT  {vj}  *+  Vt 2)  (4-26) 
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u(7)  =  VW  =  +  ajZ"1  +  a2Z~2  + . . .  avZ~v 

U(Z)  b o  +  ^Z-1  +  bzZ~2  + . . .  bgZ-^ 


bo  =  1. 


The  inverse  Z  transform  of  H{Z)  yields  the  impulse  response  {h/}4^  that  appears  in 
matrix  F,  equation  (3-30).  For  future  reference,  the  numerator  and  denominator  of  this 
function  will  be  identified  notationally  as 


H(Z)  -  ms. 

mz)~Wr 


(4-27) 


The  composite  filter  described  by  the  transfer  function  H(Z)  may  be  protected,  in 
an  inefficient  way,  by  defining  the  parity  generation  and  regeneration  subassemblies 
shown  in  the  respective  parts  of  Figure  4-2.  For  the  parity  generation  process,  each 
parity  channel  in  Figure  4-2a  represents  the  combined  effects  of  the  transfer  function 
H(Z)  and  the  respective  parity  weighting  jq^j  ,  equation  (2-9),  defining  the  0th 
parity  position  for  the  code.  The  Z  transform  of  the  c^h  parity  channel  will  be  denoted  by 
QJZ). 


Qc(Z )  = 


c  «  0, 1, ....  (n  -  k  -  1) . 


(4-28) 


The  decimator  at  the  output  of  each  channel  is  denoted  by  ik  indicating  that  only  one 
sample  is  used  for  each  k  input  samples  [30]. 

As  was  shown  earlier,  fault-tolerance  is  achieved  by  recomputing 
corresponding  parity  values  from  the  filter  system  outputs.  Any  discrepancies,  within 
the  roundoff  tolarance  of  the  system  and  the  error-detecting  power  of  the  code, 
indicates  a  protected  malfunction  somewhere  in  the  system,  including  even  the 
subsystems  generating  or  regenerating  the  parity  values.  The  parity  estimation 
subassembly,  outlined  in  Figure  4-2b,  employs  the  usual  FIR  filter  channels  defined  by 
the  optimum  estimator,  equation  (3-42),  assuming  all  bias  terms  are  zero. 

Of  course,  this  system  configuration  is  grossly  inefficient  since  it  replicates 
(n  -  k)  actions  of  the  original  filter,  one  in  each  parity  channel.  Even  the  decimated 
sampling  at  reduced  rate  ^  does  not  mitigate  the  increased  complexity.  However, 
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PARITY  GENERATION 


INPUT  I 

_ I 


(n-k) 

PARITY 

VALUES 


DENOTES 

DECIMATION 


PARITY  GENERATION 

FIGURE  4  -2a 


PARITY  CALCULATIONS  TO  PROTECT  TRANSFER  FUNCTION 
WITH  CONVOLUTIONAL  CODE 

FIGURE  4  -2 
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PARITY  REGENERATION 

FIGURE  4  -2b 


PARITY  CALCULATIONS  TO  PROTECT  TRANSFER  FUNCTION 
WITH  CONVOLUTIONAL  CODE 
FIGURE  4  -2 
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other  steps  can  drastically  reduce  this  complexity  bringing  the  implementation  in  the 
realm  of  practical  applicability. 

A  framework  for  modifying  rate  -  convolutional  codes  while  still  preserving  the 
code’s  original  distance  properties  will  be  explored  here.  Algorithms  for  computing  the 
distance  of  a  convolutional  code  [15,31,32]  examine  the  distance  contribution  resulting 
from  each  group  of  k  input  digits  as  mapped  by  a  typical  n  columns  of  the  generator 
matrix  G,  equation  (2-6).  The  respective  columns  and  rows  affecting  a  generic  n  code 
digits  are  given  by  matrix  G^. 


G(m) 


r0  Qm' 

0  Qm- 1 
o  I 
I  I 
I  I 

0  Q , 

Jk  Qq  j 


(m  +  i)kxn  Code  Segment  Matrix. 


(4-29) 


The  right  (n  -  k)  cilumns  form  the  O  matrix,  equation  (2-8),  which  contains  the 
quantities  that  are  to  be  modified. 

The  minimum  distance  profile  of  a  convolutional  code  relies  on  the  interaction  of 
k  input  digit  groups  with  the  row  space  of  G(m).  Hence,  if  a  new  matrix  G^  containing 
a  different  Q'  still  has  a  combinatorially  equivalent  row  space  of  the  original  G(m), 
equation  (4-29)  above,  the  minimum  distance  properties  remain  the  same  [15,31,32]. 
The  rightmost  columns  Q '  of  the  code  segment  defining  the  modified  code  G^ 

corresponds  to  {n-k)  new  FIR  parity  channel  filters  designated  O'0(2),  0^(2) . 

QVfr_i(2).  (See  similar  equations  (2-8)) 


Q'(m)  = 


{Ik 


(4-30) 
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SIMPLIFIED  PROTECTION  USING 
MODIFIED  CONVOLUTIONAL  CODE 

FIGURE  4  -3 


However,  if  each  of  the  new  parity  channel's  transfer  function  contains  D(Z),  the 
denominator  of  H(Z),  equation  (4-27),  the  poles  are  effectively  removed  from  the  parity 
generation  filters,  and 

Q'C(Z)  =  D(Z)  RC(Z) 


Implies 

H(Z)OHZ)  =  RC(Z)  N(Z)  =SC{Z) 


(4-31) 


c=0, 1,2, ....  (n-  k- 1) . 

The  parity  channel  only  needs  to  implement  the  FIR  filter  described  by  the 
transfer  function  SC(Z).  In  addition,  the  decimation  operation  ik,  commutes  with  such 
filter  structures  [30].  Figure  4-3  shows  the  theoretical  impact  of  this  simplification  for 
typical  channel  c=  0, 1, ....  (n-  k- 1)  .  The  theory  achieving  these  simplifications  will 
be  developed  next. 

The  columns  of  G(m )  may  be  viewed  as  vectors  from  a  complex  vector  space,  V 
with  dimension  [(m  +  1  )k].  Associated  with  every  vector  space  is  its  dual  space  [24, 
Chapt.  Ill,  Section  6],  a  space  of  linear  functionals  from  V  into  the  complex  numbers. 
The  dual  space  of  V  is  written  as  V *  and  its  members  are  also  written  as  column 
vectors.  The  column  space  of  G(m)  is  a  subspace  of  V  which  is  also  uniquely  specified 
by  elements  in  the  dual  space.  The  subspace  of  V*  which  annihilates  the  column 
space  generated  by  G(m)  is  called  the  annihilator  of  this  space,  Anr{G(ml\. 

Anii[G(ml\  c  V* 

a  e  Anr{G(m)\  < — » a?  &m)  =  o 

(4-32) 

SLis[(m+  t)k\kxi  Vector 
aT  Denotes  Transpose 

The  matrix  whose  linearly  independent  columns  span  Anri[G(m)\  is  denoted  by  H(mK 

In  order  to  find  the  annihilator  space  generated  by  H(m),  matrix  G(m)  is  reduced 
by  a  series  of  column  operations  and  row  permutations  to  a  canonical  form  for  which 
the  annihilator  has  an  obviously  simple  configuration.  The  row  permutations  are 
represented  by  a  nonsingular  matrix  D  which  in  effect  permutes  the  basis  for  V. 
Because  of  the  limited  type  of  operations  represented  by  D,  it  has  the  property: 

cr1  .  Dr 
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The  column  operations  are  represented  by  the  nonsingular  nx  n  matrix  C.  The 
resultant  canonical  form  for  G(m )  is  given  below  where  the  ln_k  matrix  is  a 
consequence  of  the  fact  that  G(m)  is  a  mapping  ONTO  the  n-tuples  representing  each 
codeword  symbol. 


DG{m)C  = 


(0  S  > 
0  ln-k 

Ik  0  J 


C, nxn  Non  singular 

D,  ( m  +  1)/c  x(m  + 1  )k  Permutation 


(4-33) 


Sis((m+  1  )/c  —  n)  x  ( n-k ) . 


The  matrix  representing  the  annihilator  of  this  canonical  form  is  easily  written: 

(l(m+i)Ac— n  I  |  0)  where  submatrix  0  is  [( m  +1)k  -n]xk. 

However,  in  the  original  basis  the  annihilating  matrix  for  G(m)  is  given  by  matrix,  H(m)T. 


(4-34) 


Then  the  usual  dual  space  identity  applies  to  any  equivalent  code  segment  generating 
matrix  JmK 

H(m)TJM  sO  0  is[(m+  1)k—  n]xk  Zero  Matrix .  (4-35) 

The  columns  of  H(m)  generate  the  column  space  of  Anri[G(m^\  because  the  matrix  DT  is 
nonsingular  due  to  its  permutation  origins. 


Anri[G(mi]  =  Column  Space  H(m) 


hW  -  dt 


(l(m+i)k-n\ 
-ST 
0 


(4-36) 


0  is  k  x  [(m  +  1  )k  -  n] 


A  generator  matrix  with  a  different  set  of  (n  -  k)  parity  columns,  O'  similar  to 
those  in  equation  (2-8)  must  satisfy  the  equality 
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H(m)T 


) 


(4-37) 


This  guarantees  that  the  columns  of  Q'  generate  a  code  segment.  These  equations 
can  be  separated  into  two  parts  with  O'  appearing  in  only  one  equation. 

Q'  =  0.  (4-38) 


The  new  columns  in  O'  must  also  eliminate  the  poles  in  D(Z)  to  simplify  each  parity 
filter  channel.  This  constraint  may  be  expressed  in  terms  of  two  unknown  quantities  R 
and  Q'  where  is  the  aggregate  of  the  FIR  filters  described  by  transfer  functions  RJ,Z), 
c  =  0, 1 . (n  -  k  - 1 ).  On  the  other  hand,  a  matrix  A  contains  the  effects  of  D(2). 


AR  =  Q' 


(4-39a) 


'  1  0  0  0  0 

bj  1  0  -  - 

M  I  I  I 

b^b2--~ 
---00 
---10 
-  -  -  bx  1 

^5  6&-1  ~  tyz.  b\ 
0  t*  |  |  bz 

0  0  -  -  | 

I  I  I  65  - 

,0  0  0  0 


*0,0  *1,0 
*1,0 
*2,0 


l^*(W-8-1),0  *(Af-S-1),1 


*0,(n-/c-1) 

*1,(n-k-1) 


*(M-8-1),(n-fc-1)J 


(4-39b) 
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;  M  =(m  +  1)/c. 


1,0  Q'm-w  ~  J 

The  annihilator  H(m)T  reduces  this  to  a  homogeneous  system  when  the  additional 
constraints  in  equation  (4-38)  are  imposed. 

H(m)TAFI=  0.  (4-40) 


This  system  will  provide  dependencies  among  the  components  of  Ft. 

The  combined  effects  of  the  annihilator  restriction  and  the  poles  of  H(Z)  appear 
in  the  matrix  product,  giving  the  (( m  +  1  )k-  n]  x  [(m  +  1  )k  -  8]  matrix  A. 


A  =  H(m)r  A . 


(4-41) 


A  series  of  row  operations,  represented  by  nonsingular  matrix  £  brings  A  to  echelon 
form  where  the  row  rank  is  denoted  by  p. 


E  A 


r  \ 

I  B  m 

-  .  -  Asq/S  pxp 

0  |  o  B  is  p  x[(m  +  1)k  -  &  -  p] 


(4-42a) 


p  £  MIN  {[(m  +  1  )k  -  n],  {(m  +  1  )k  -  8]} .  (4-42b) 

Since  E  is  nonsingular,  the  homogeneous  equation  (4-40)  is  equivalent  to: 


(A^QRnO. 


(4-43) 
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Constraints  among  components  in  the  unknown  matrix  R  follow  from  this  equation. 
The  components  in  R  may  be  separated  into  independent  and  dependent  parts 
following  the  partition  of  the  matrix  in  equation  (4-42a). 


px(n-/r) 

'rd' I  Dependent  Part 

K  Rl  J  [(m  +  ij/c  -  8  -  pj  x  (n  -  k  j 
Independent  Part 


The  upper  triangular  matrix  Asq  is  easily  inverted,  leading  to  the  dependency  between 
parts. 

RD  =  AsqBR,.  (4-45) 


Various  choices  for  Ri  provide  potential  solutions  for  the  pole-canceling  parity 
channels  represented  by  Q'. 


( ~  1 
A-1 


BR, 


=  O' 


Q'  is  (m  +  1)/c  x  (n  -  k) . 


(4-46) 


The  new  choices  for  a  code  generating  segment  are  contained  in  the  matrix, 


Q'im)  m 


r0  1 

r  „  „  \ 

-  1  A 

“A gq  B  R/ 

\  / 

A  1 

(4-47) 


There  is  no  guarantee  that  any  given  code  can  be  modified  to  cancel  all  the  poles  in  a 
selected  filter  even  if  the  respective  dimensions  are  selected  large  enough.  The  row 
rank  p  of  A,  an  [( m  +  1  )k  -  n]  x  [(m  +  1  )k  -  5]  matrix  determines  the  number  of 
independent  choices  in  R  which  in  turn  directly  impacts  the  new  filter  weights  in  O',  a 
(m  +  1)*  x  (n  -  k)  matrix  which  totally  defines  the  code  segment  G ^  because  of  its 
systematic  form. 
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A  simple  example  employing  one  parity  channel  is  given.  The  transfer  function 
for  a  7th  order  elliptic  filter  is 

W(Z)=  100(0.322928-0.382389Z~1+0.669801Z~2-0.061Z~3-0.062693Z~4+0.66983Z~5-0.381679Z~6+0.322016Z-7) 
1—5.1 1 6546Z- Vl2.1 91 41 8Z~2— 1 7.2631 71Z-3+15.595655Z-^-8.963986Z~®+3.035586Z-6— 0.4687S7Z- 7 

This  filter  will  be  protected  with  a  rate  4/5  binary-based  convolutional  code  taken  from 
a  table  [33].  Its  parity  channel  filter  has  Ztransform  QC(Z). 

QJZ)  =  1  +  Z"1  +  Z"2  +  Z"3  +  Z-4  +  Z-5  +  Z-8  +  z-10  +  Z~u  +  Z“12  +  Z“15 

This  is  a  systematic  rate  4/5  code  and  has  a  minimum  column  distance  function  of  3. 
Thus,  it  can  detect  any  three  errors  in  a  data  block,  where  a  block  consists  of  four  data 
words  and  one  parity  word. 

After  altering  Q(Z)  to  cancel  the  poles  of  H(Z),  the  following  Q'C{Z)  is  chosen: 

Qc(Z)  =  0.5631  +  0.3926Z-1  -  1.2983Z-2  -  0.7239Z-3  +  0.9201 Z-4  + 15274 Z-5 

-  0.8176  Z"8  +  0.3971Z-10  -  1.4297Z"11  -  0.1094Z"12  +  0.5869Z-15 
The  product  of  H(Z)  and  Q(Z)  results  in 

SJZ)  =  0.3491  +  0.5481  Z"1  -  1.2378Z^2  +  0.9832Z^3  +  0.1847 Z"4  -  0.2687Z"5 
+  0.8345Z-6  +  0.2387Z"7  -  0.7845Z^  +  0.9276Z"9  -  0.4218^°  -  0.1793Z"11 
+  0.2947Z"12  -  1.3923Z"13  -  0.5782Z"14  +  0.8383Z^15  . 

The  impact  of  the  code's  parity  filter  responses  on  the  MSE  is  quite  complex. 
However,  as  equation  (4-46)  shows,  there  is  a  direct  relationship  between  the  choices 
of  the  independent  part  of  the  modified  parity  filter  and  the  actual  parity  weighting 
values  in  Q\  This  may  be  incorporated  into  the  zf  expression  for  the  optimum  MSE, 
equation  (3-43).  A  typical  term  in  e2  containing  a  bilinear  form  in  reduces  to  a 
sum  of  squares  when  the  noise  components  are  assumed  uncorrelated.  This  is 
analogous  to  the  earlier  case  for  cyclic  codes,  equations  (4-13)  through  (4-15).  A 
similar  result  for  the  case  of  a  single  parity  channel  has  a  typical  term: 
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The  inuividual  noise  powers  are  denoted  by  the  of  terms. 

The  influence  of  the  code  modification  choices  as  expressed  in  equation  (4-46) 
may  be  incorporated  in  this  last  equation.  However,  unlike  the  situation  for  cyclic 
codes,  a  discrete  Fourier  transform  does  not  seem  naturally  applicable.  In  general, 
convolutional  codes  are  not  based  directly  on  roots  associated  with  the  parity  channel 
weighting.  This  is  an  important  unsolved  problem.  Nevertheless,  some  form  of 
transform  may  help  decompose  the  problem  further.  There  is  an  annihilator  subspace 
affiliated  with  finding  the  modified  parity  channel  weightings. 

There  are  several  other  construction  techniques  for  real  convolutional  codes 
besides  the  direct  translation  of  binary  codes  to  the  real  field.  The  rate  1/2  code 
constructed  by  Mathys  [14]  is  one  example.  Another  larger  class  of  codes,  to  be 
described  briefly  below,  is  based  on  a  technique  due  to  Wyner  [13,  Sect.  13.3]  and 
employs  cyclic  codes  as  a  basis.  The  codes  will  be  developed  in  their  nonsystematic 
form  for  ease  of  exposition;  it  is  a  straightforward  matter  to  transform  them  to  a 
systematic  format. 

The  Wyner  real  code  has  a  free  distance  of  six  or  more  with  redundancy  of  3 
positions  (n  -  k  =  3)  and  constraint  parameter  m  =  1 .  The  code  length  n  is  a  design 
variable  so  that  the  rate  is  (1  -  3 In).  The  subblocks  of  the  generator  matrix  G,  equation 
(2-6)  containing  a  constraint  length  of  the  code,  will  be  denoted  by  G^.  It  in  turn  fully 
defines  the  two  subblocks  G0  and  Gv 


2kx2n  Constraint  Length  Subblock. 


(4-49) 


The  construction  centers  on  the  form  of  the  parity-check  matrix  HW  associated  with 
Gt1).  This  parity-check  matrix  contains  two  appropriately  sized  subblocks  H0  and  Hy 
that  will  be  detailed  fully  in  a  moment. 


;  2 (n  -  k)  x  Hj  Submatrix 


n 


Q ' 
o 

i 


HqJ 

1  Denotes  1  x  n  vector  of  all  Ones 
0  Denotes  1  x  n  vector  of  all  Zeros . 


(4-50a) 


G^)  =  0, 

where  h  Denotes  the  conjugate  transpose. 


(4-50b) 


The  submatrices  H0  and  are  defined  by  a  real  cyclic  code  employing  nth 
complex  roots  of  unity. 
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{5  x  n  Matrix  Defines  Real  BCH  Code  with  dmtn  =  6.} 


(h  )  {3  *  n  Matrix  Defines  Code  Containing  jF^j  with  dm-,n  -  4.J 


The  proof  that  this  does  have  at  least  a  minimum  free  distance  of  six  may  be  found  in 
Peterson  and  Weldon  [13,  Sect.  13.3].  The  systematic  form  can  be  derived  by 
applying  row  manipulations  to  H^1)  resulting  in  corresponding  part  with  a 
characteristic  separation  of  parity  part  submatrices  P0  and  Pv 


-Po  '3 

-p?  0 


0  o' 

-Po  i3, 


Pi  kx(n-k)  Matrix 
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gd)  „  '*  po  0  P1 

s  [o  o  ik  p0 


k  =  n-3. 


An  example  of  the  Q  submatrix  part  of  a  systematic  version  of  G  can  be  generated  by  a 
simple  computer  program. 


Q  = 


'40.0000006+00  -17397396-01  +8.2717576-01 
+0.0000006+00  -2.4594476  +  00  -2.9565236  +  00 
+0.0000006  +  00  -6.283361e  +  00  -5.1432276  +  00 
+0.0000006  +  00  -6.1 8681 3e  + 00  -3.4975676  +  00 
+8.4105016-02  -1.2730876  +  00  +18898226+00 
+1.8898226-00  -3.8606016  +  00  +2.6716196  +  00 
+2.3565706  -  00  -5.8140916  +  00  +4.5784346  +  00 
+18898226-00  -5.662539e  +  00  +4.4735576+00 


n  =  7  k= A  m=1 
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